Constructive Neural Network Construction Methods

This paper presents an overview of constructive neural networks. In the constructive method, training starts with minimal structure and then more layers of neurons are added according to some rule which is predefined. Constructive algorithms provide a natural framework for incorporating problem-specific knowledge into initial network configurations. In this paper we have enlightened various constructive algorithm methods that construct feed forward architecture for regression problems. Cascade Correlation trains one unit at a time instead of training the whole network at once. Recurrent Cascade Correlation is a recurrent version of the Cascade Correlation learning architecture. Dynamic Node creation algorithm is a valuable tool for finding backpropogation architectures for arbitrary mapping problems. Constructive Back propogation benefits from simpler implementation and the ability to utilize stochastic optimization routines. Fixed Cascade Error is an enhanced version of Cascade Error. Adaptive sigmoidal function results in increased flexibility, smoother learning, and a better generalization performance. Decay RBFs can uniformly approximate any continuous multivariate functions with arbitrary precision without training. C-Mantec is a novel neural network constructive algorithm generates very compact neural architectures with state-of-the-art generalization capabilities. CFN is a novel scalable learning system for tackling massive amount of data.